Distance estimation using multi-camera device

ABSTRACT

A method and apparatus for measuring a distance to an object using a device (100) having at least two cameras (104, 106) is described. One or more first images including the object are acquired by a first camera of the device and one or more first reference images are acquired by a second camera of the device, while the device is in a first position. One or more second images including the object and one or more second reference images are acquired by cameras of the device, while the device is in a second position, different from the first position. Based on the first and second reference images, information of the displacement of at least one camera of the device between the first and second positions are determined. The distance from the device to the object is calculated based on the first and second images including the object and the determined information on the displacement of the at least one camera.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is the United States National Phase of PatentApplication No. PCT/IB2014/060462 filed 6 Apr. 2014, which claimspriority to United States Provisional Nos. 61/809,447 filed 8 Apr. 2013and 61/809,464 filed 8 Apr. 2013, each of which is incorporated hereinby reference.

FIELD OF THE INVENTION

The present invention relates to distance measurement and in particularto distance measurement by portable devices.

BACKGROUND OF THE INVENTION

Mobile devices, once used solely for telephone communications, are usedfor a wide range of tasks and include various apparatus.

Korean patent publication 2013/0022831 to Ryoo describes a method ofmeasuring distance, height and length of an object using a mobilecommunication terminal.

Chinese patent publication 202940864 to Wu Hao describes incorporating adistance sensor into a mobile phone.

Chinese patent publication 10334213 to Liu Guohua describes measuringdistance using two cameras with a known distance between them.

These devices either lack sufficient accuracy or require additionalhardware not included in many existing mobile devices.

SUMMARY OF THE INVENTION

An aspect of some embodiments of the present invention relates to usinga mobile device with a front camera and a rear camera to calculate thedistance to an imaged object.

The distance to an object is optionally calculated by achieving twoobject images and two corresponding reference images, each pair ofcorresponding images being acquired from a different position. Thedisparity and/or distance in pixels and/or in the real world, betweenthe different positions is determined, and accordingly the distance iscalculated by triangulation.

There is therefore provided in accordance with embodiments of thepresent invention, a method of measuring a distance to an object, by adevice having at least two cameras, comprising acquiring one or morefirst images including the object by a first camera of the device andone or more first reference images by a second camera of the device,while the device is in a first position, acquiring one or more secondimages including the object and one or more second reference images, bycameras of the device, while the device is in a second position,different from the first position, determining, based on the first andsecond reference images, information on the displacement of at least onecamera of the device between the first and second positions, andcalculating the distance from the device to the object, based on thefirst and second images including the object and the determinedinformation on the displacement of the at least one camera.

Optionally, acquiring the one or more second images including the objectcomprises acquiring by the first camera, through which the one or morefirst images of the object were acquired. Optionally, acquiring the oneor more first images including the object comprises acquiring aplurality of images while the device is in the first position andgenerating a single image including the object from the plurality ofacquired images. Optionally, generating a single image including theobject from the plurality of acquired images comprises selecting one ofthe images.

Optionally, generating a single image including the object from theplurality of acquired images comprises generating a combination of someor all of the images and/or automatically analyzing the quality of theacquired images and discarding images of a quality below a giventhreshold. Optionally, the method includes displaying to a userinstructions directing to change the position of the device to thesecond position, after acquiring the one or more first images.Optionally, displaying the instructions comprises displaying anindication of an extent of matching of a current image acquired by thesecond camera, to the first reference image. Optionally, the methodincludes displaying, to a user, a video stream acquired by the secondcamera, after acquiring the one or more first images, so that the usercan use the video stream in directing the device to the second position.

Optionally, determining information on the displacement of the at leastone camera of the device comprises comparing the size of a body organshown in both the first and second reference images. Optionally,determining information on the displacement of the at least one cameraof the device comprises comparing the size of a cornea shown in both thefirst and second reference images. Optionally, determining informationon the displacement of the at least one camera of the device comprisesdetermining an extent of relative rotation between the first and secondreference images.

Optionally, determining information on the displacement of the at leastone camera of the device comprises determining based on a known distancebetween the first and second cameras. Optionally, the first and secondcameras are on opposite sides of the device. Optionally, the one or morefirst images including the object and the one or more first referenceimages are acquired in pairs of reference and object images, the imagesof each pair being acquired substantially concurrently within less than1 second.

There is further provided in accordance with embodiments of theinvention, a device, comprising: a housing, a first camera included inthe housing, a second camera included in the housing; and a processor,included in the housing, configured to receive one or more first objectimages including an object, acquired by the first camera, and one ormore second object images including the object, to receive one or morefirst reference images acquired by the second camera and one or moresecond reference images, to determine a displacement of at least onecamera of the device between acquiring the first object image andacquiring the second object image, responsively to the first and secondreference images, and to calculate the distance from the device to theobject based on the first and second object images and the determineddisplacement.

Optionally, the first and second cameras are on opposite sides of thehousing. Optionally, the processor is configured to control the firstand second cameras to acquire the images in pairs of concurrentlyacquired object images and respective reference images. Optionally, thedevice includes a screen located on a same side of the housing as thesecond camera. Optionally, the processor is configured to provide, afterreceiving the one or more first object images and first referenceimages, instructions on changing the position of the housing, foracquiring the second object and second reference images. Optionally, theprocessor is configured to analyze an input stream provided by thesecond camera relative to the one or more first reference images, afterreceiving the one or more first object images and first referenceimages, in order to determine when the housing is in a position suitablefor acquiring the second reference images and the second object images.

Optionally, the device includes a screen and wherein the processor isconfigured to display on the screen, after receiving the one or morefirst object images and first reference images, a stream of imagesacquired by the second camera, in order to allow a user to position thehousing for acquiring the second images.

Optionally, the processor is configured to determine an extent ofrelative rotation between the first and second reference images and usethe determined extent of relative rotation in determining thedisplacement of the at least one camera of the device between acquiringthe first object image and acquiring the second object image.

Optionally, the processor is configured to determine the displacement bycomparing the size of a body organ shown in both the first and secondreference images.

There is further provided in accordance with embodiments of theinvention, a computer program product for distance estimation comprisinga computer readable non-transitory storage medium having computerreadable program code embodied therewith, which when executed by acomputer performs, receives one or more first images including an objectand one or more first reference images, receives one or more secondimages including the object and one or more second reference image,determines, based on the first and second reference images, informationon a displacement of at least one camera of a device between acquiringthe one or more first and one or more second images, and calculates adistance from the device to the object, based on the first and secondimages including the object and the determined information on thedisplacement of the at least one camera.

BRIEF DESCRIPTION OF FIGURES

Exemplary non-limiting embodiments of the invention will be describedwith reference to the following description in conjunction with thefigures. Identical structures, elements or parts which appear in morethan one figure are preferably labeled with a same or similar number inall the figures in which they appear, in which:

FIG. 1 is a schematic illustration of a mobile device 100 which may beconfigured to operate in accordance with an embodiment of the invention;

FIG. 2 is a flowchart of acts performed in estimating a distance to anobject, in accordance with an embodiment of the invention;

FIG. 3 is a schematic illustration of the mobile device in a rotatedorientation relative to the orientation of FIG. 1, in accordance with anembodiment of the invention; and

FIG. 4 is a schematic diagram of a calculation of the distance to anobject based on images acquired from two different positions, inaccordance with an embodiment of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

An aspect of some embodiments of the invention relates to a method ofmeasuring a distance to an object by a mobile device including at leasttwo cameras. At least two images including the object are acquired fromrespective different positions of the device. Substantially concurrentlywith acquiring the images including the object (referred to herein as“object images”), respective reference images are acquired by adifferent camera than the camera acquiring the respective object image.The reference images are used to determine the relative positions of thecamera(s) when the at least two images of the object were acquired andusing the at least two images and the relative positions of the cameras,the distance to the object is determined.

In some embodiments, the different positions of the cameras acquiringthe images are achieved by rotating the device and positioning thecamera acquiring the reference images such that the reference images aresubstantially identical. The relative position of the camera acquiringthe object images can then be determined from the extent of rotation andthe positions of the cameras on the device.

In other embodiments, the different positions of the cameras acquiringthe images are achieved by changing the distance between the device anda user holding the device. The distances to the user holding the deviceis determined from the reference images.

The term device position is used herein to relate to both the locationand orientation of the device, such that two different orientations ofthe device, even if the device is in the same location, are referred toherein as two different positions.

In some embodiments of the invention, the object images are all acquiredby a first camera and the reference images are all acquired by a secondcamera. In other embodiments, however, a same camera of the device isused for one or more of the object images and one or more of thereference images.

Overview

FIG. 1 is a schematic illustration of a mobile device 100 which may beconfigured to operate in accordance with an embodiment of the invention.Device 100 optionally comprises a touch screen 102, a front camera 104,having an imaging axis 134, on the same side as the touch screen 102 anda rear camera 106, having an imaging axis 132, directed in an oppositedirection from front camera 104. A processor 110 and a memory 112 areembedded within mobile device 100. Processor 110 is configured withsoftware to carry out distance estimation based on images acquired byfront camera 104 and rear camera 106 as described herein below. Device100 optionally comprises a smart-phone or tablet, although other deviceshaving the elements discussed herein may also be used. Generally,processor 110 is configured to perform various tasks other than distanceestimation, such as encoding and decoding speech signals, and/or anyother tasks known to be performed by mobile devices.

The software may be downloaded to processor 110 in electronic form, overa network, for example, and stored in memory 112. Alternatively oradditionally, the software may be held on tangible, non-transitorystorage media, such as optical, magnetic, or electronic memory media.Further alternatively or additionally, at least some of the functions ofprocessor 110 may be performed by dedicated or programmable hardwarelogic circuits.

FIG. 2 is a flowchart of acts performed in estimating a distance to anobject, in accordance with an embodiment of the invention.

When mobile device 100 is instructed (204) to estimate a distance to anobject, rear camera 106 is aimed at the object and mobile device 100optionally begins to execute a software process that manages a distanceestimation procedure. At the beginning of the estimation procedure,mobile device 100 acquires (206) one or more images through rear camera106, which is aimed at the object, and in addition acquires (208) one ormore images through front camera 104. Optionally, a single image pair,referred to herein as (Rear1, Front1), is selected (210) to representthe rear camera and front camera images.

Mobile device 100 then optionally instructs (212) the user to change theposition or pose of mobile device 100 to a new position and aim frontcamera 104 in the same direction as before the change in position.Mobile device 100 optionally verifies (214) that front camera 104 isproperly aimed and then acquires (216) one or more images by both thefront and rear cameras. Mobile device 100 optionally selects (218) asingle image acquired from the new position for each of the front andrear cameras. The selected pair of images from after the position changeis referred to herein as (Rear2, Front2).

Mobile device 100 analyzes the selected images acquired before (Rear1,Front1) and after (Rear2, Front2) the position change, to determine(222) the change in position of the rear camera 106 due to the change inthe position of mobile device 100. Based on the determined change in theposition in the rear camera and the selected images of the object (Rear1and Rear2) from before and after the change, the distance to the objectis estimated (224), as detailed hereinbelow.

Alignment Instructions

In some embodiments of the invention, the instructions (212) to the userto change the position or pose of device 100 include a display of acurrent video stream acquired by front camera 104, allowing the user toadjust the position of front camera 104 according to the images it iscurrently acquiring.

Optionally, the instructions (212) to the user further include theselected (210) image acquired by front camera 104 before the positionchange (Front1), so the user can easily match the current position offront camera 104 to the selected (210) previously acquired image Front1.Optionally, the current video stream is overlaid on the displayed imageFront1 to allow simple alignment by the user. Alternatively, the currentvideo stream may be displayed side by side with image Front1, possiblywith a suitable grid correlating between the images. Furtheralternatively or additionally, rather than displaying the Front1 image,a shadow or grid extracted from the Front1 image is displayed, and theuser is requested to match the current video stream to the displayedshadow or grid.

Alternatively or additionally, processor 110 compares the images of animage stream currently collected by front camera 104 and the selectedFront1 image, for example using feature matching, and accordinglyprovides aiming instructions, for example in the form of displayedarrows and/or other graphical guidance or spoken directions. The featurematching optionally includes selecting a feature appearing in bothFront1 and the images of the currently collected image stream anddetermining the relative positions of the features in Front1 and theimage stream.

In some embodiments of the invention, processor 110 calculates anumerical matching extent score and displays the calculated matchingextent score to the user. Alternatively or additionally, processor 110provides a stop instruction when the matching extent is above apredetermined threshold. The threshold may be a preconfigured thresholdused universally, or may be a dynamically adjustable threshold accordingto a required accuracy level of the distance determination. In someembodiments of the invention, a relatively low threshold is required andprocessor 110 digitally compensates for the mismatch by performing imagerectification, as described hereinbelow.

Optionally, the instructions (212) also request the user to remain stilluntil the images are acquired, in order to minimize the position changesbetween acquiring (Rear1, Front1) and (Rear2, Front2).

Initiation and utilization

The estimated distance is optionally provided to the user by display onscreen 102, sounding to the user or any other suitable method.Alternatively or additionally, the distance may be used by processor 110or provided to a different processor, for further calculations. Based onthe distance to the object, processor 110 may optionally determine othersizes and/or distances in the scene, such as the size of the objectand/or the distances and sizes of other objects in the images. Thus,device 100 may operate as a measurement tool. Alternatively oradditionally, the distance estimation is used to apply scale-inaugmented reality applications, for example in overlaying virtualobjects on an image, stream of images or live scene. In some embodimentsof the invention, the object size is used by processor 110 to improveobject tracking between image frames in a video stream acquired bymobile device 100, for example in surveillance applications.

The instruction (204) to estimate a distance to an object is provided,in some embodiments, by a human user, who is aware of the estimation andparticipates knowingly in the process. In other embodiments, theinstruction (204) to estimate a distance is provided automatically byanother software running on device 100. In these embodiments, the usermay participate knowingly in the process or may not be aware of theprocess. For example, the instruction (212) to move the device to a newposition may be provided among other instructions provided by the othersoftware initiating the distance estimation, such that the user does notnecessarily know that the instructions are performed for distanceestimation. Alternatively or additionally, the instruction (212) tochange the position of device 100 is provided in an implied manner,without the user necessarily understanding that there was aninstruction. For example, an image on screen 102 may be rotated in orderto cause the user to instinctively rotate device 100.

Acquiring of Images

Referring in more detail to acquiring (206, 208, 216) images throughfront camera 104 and rear camera 106, in some embodiments of theinvention, at least three, at least five or even at least ten images areacquired by each of the cameras in each of the acquiring sessions (e.g.,before and after the position change). In some embodiments of theinvention, the quality of the images are assessed as they are acquiredand the image acquiring is repeated until an image, a predeterminednumber of images or pairs of front and rear images considered to have asufficient quality, are acquired. Accordingly, in these embodiments, thenumber of acquired images varies according to the quality of theacquired images. Optionally, the same number of images is acquired bythe front and rear cameras. Alternatively, more images are acquired bythe rear camera 106, as the chances that the images of the rear cameraare blurred or otherwise unsuitable are generally relatively higher thanthe chances that the front camera images are blurred. It is noted,however, that in cases in which front camera 104 is expected to havehigher chances of providing unsuitable images, front camera 104 may beinstructed to acquire more images than rear camera 106.

Optionally, in cases in which the front and rear images are acquired inpairs, the front and rear camera images of each pair are acquiredsimultaneously or within a very short time difference, such as withinless than 5 seconds, less than 1 second, less than a half a second,possibly even less than 30 milliseconds.

Optionally, the same criterion is used in determining the number ofimages acquired before and after the position change of device 100. Insome embodiments, in both image acquiring sessions, images are acquireduntil a predetermined number of images of sufficient quality wereacquired. Alternatively, the same number of images is acquired beforeand after the position change. In other embodiments, more images areacquired before the position change or after the position change. Insome embodiments of the invention, the quality assessment after theposition change includes in addition to an objective quality, or insteadof an objective quality, a matching measure indicative of the extent towhich the image matches the images acquired before the position change.

Optionally, the image selection (210) of (Rear1, Front1) is performedbased on quality assessment methods, such as verifying that the imagehas strong gradients and/or that the camera did not move while acquiringthe picture (e.g., based on accelerometer measurements and/or imageanalysis). Optionally, the selection includes selecting Rear1 as thebest quality image acquired before the position change by rear camera106, and selecting the corresponding front camera image acquired at thesame time to be Front1. Alternatively, after selecting Rear1, Front1 isselected among a limited number of images acquired within a short periodof time before, in parallel to and/or after acquiring Rear1.Alternatively, the selection of Rear1 and Front1 is based on a weightedscore giving different weight to the quality of the rear image and tothe quality of the front image. In some embodiments, the time differencebetween acquiring each pair of images is also factored into the score.

Alternatively or additionally, the selection provides a combination oftwo or more images, for example generated as a weighted sum of theimages or of those images considered having a quality above a giventhreshold. Optionally, low quality images (e.g., blurred images) arefiltered out, and the weighted selected images are calculated only basedon the images considered of sufficient quality.

The image selection (218) of (Rear2, Front2) may be performed in amanner similar to the selection (210) of (Rear1, Front1). Alternatively,Front2 is selected as the first image considered sufficiently aligned toFront1, which has a sufficient quality.

In some embodiments, processor 110 determines the location of the userrelative to the background in the Front1 selected image, and analyzesthe images acquired after the position change based on the extent towhich the location of the user relative to the background in theseimages, matches the relative location in Front1. Optionally, the imageacquired after the position change, which best matches Front1 isselected. Alternatively, analyzed images having a difference in locationbeyond a predetermined value are weeded out from consideration. Furtheralternatively, the extent to which the relative location of the user inthe analyzed images matches that in Front1 is weighted into a qualityscore used in selecting Front2.

Optionally, if an image of sufficient quality and/or matching is notfound among the images acquired after the position change, the user isrequested to repeat the acquiring (216) of images after the positionchange or is requested to repeat the entire procedure of FIG. 2.

In some embodiments of the invention, processor 110 also verifies thatthe axis of the camera as determined from the rear camera images and/orfront camera images did not change substantially between the acquiringof images before and after the position change. Alternatively oradditionally, rectification is performed to compensate for movement ofthe camera axis, as detailed hereinbelow.

Distance Estimation

The estimation (224) optionally includes image rectification to correctfor rotation bias of the images acquired before and after the positionchange of rear camera 106 (Rear1, Rear2). In the image rectification,Rear1 and Rear2 are adjusted such that they are both in the same plane.The rectification is performed using any suitable method known in theart, such as any of the methods described in US patent publication2013/0271578, to Richards, US patent publication 2005/0180632 to Aradhyeet al. and Lim, Ser-Nam; Mittal, Anurag; Davis, Larry; Paragios, Nikos.,“Uncalibrated stereo rectification for automatic 3d surveillance”,International Conference on Image Processing 2: 1357-1360, 2004, thedisclosures of all of which are incorporated herein by reference intheir entirety.

Processor 110 then optionally performs a triangulation method on therectified versions of Rear1 and Rear2 to determine the distance to oneor more points of interest in the images. The triangulation is performedusing any suitable method known in the art, for example, using any ofthe methods described in Richard Hartley and Andrew Zisserman (2003),Multiple View Geometry in computer vision. Cambridge University Press,the disclosure of which is incorporated herein by reference. Thetriangulation is optionally based on the distance between the positionsof the rear camera 106, before and after the position change, i.e., thepositions of rear camera 106 at the time of acquiring Rear1 and Rear2,respectively, and the f-number, also referred to as focal ratio, of rearcamera 106.

In some embodiments of the invention, estimation (224) includes creatinga stereo or disparity image based on the Rear1, Rear2 images.

Rotation

FIG. 3 is a schematic illustration of mobile device 100 in a rotatedorientation relative to the orientation of FIG. 1, in accordance with anembodiment of the invention. In the embodiments described with referenceto FIG. 3, the change in position of mobile device 100 is a rotation ofthe mobile device.

The instruction (212) to move mobile device 100 to the new positionincludes in these embodiments, an instruction to rotate the mobiledevice. Optionally, the user is instructed to position mobile device 100such that the images acquired by front camera 104 in the new position,after the rotation, are the same as the images acquired before therotation, adjusted for the extent of rotation of the mobile device 100.In some embodiments, the user is instructed to rotate mobile device 100by 180 degrees, to achieve a maximal change in the position of rearcamera 106 relative to front camera 104. In other embodiments, the useris instructed to rotate mobile device 100 by 90 degrees, or any otherdesired extent. For example, the user may be requested to rotate device100 to an extent close to 180 degrees, for example between 160-200degrees, thus allowing the user to perform the rotation quickly withoutaccurately adjusting the rotation to 180 degrees.

When rotating by 180 degrees, as illustrated by FIG. 3 which showsdevice 100 rotated by 180 degrees relative to its position in FIG. 1,the distance 332 between the imaging axis of rear camera 106 before 334and after 336 the rotation is twice the distance 140 between thelocations of front camera 104 and rear camera 106, as front camera 104remains in the same relative location before and after the rotation.

The determination (222) of the change in position of rear camera 106, isoptionally based on the known positions of cameras 104 and 106 on device100. In some embodiments of the invention, the determination includesestimation of the extent of rotation of mobile device 100, for examplebased on a comparison of the images taken by front camera 104 before andafter the rotation. Alternatively or additionally, the estimation isbased on a comparison of the images taken by rear camera 106 before andafter the rotation. Further alternatively, the extent of the rotation isknown from an angle measurement unit within device 100.

While in the above description front camera 104 is directed to the userbefore and after the rotation, in other embodiments, front camera 104 isdirected at the object whose distance is to be determined before therotation and is directed at the user after the rotation. This rotationis referred to herein as flipping. In these embodiments, after theposition change, an image 352 taken by rear camera 106 before theposition change is optionally displayed to the user, and the user isrequested to match images 354 currently acquired by front camera 104with the displayed image. The distance is optionally calculated based onan analysis of an image acquired by front camera 104 before the positionchange and an image acquired by rear camera 106 acquired after theposition change. In some embodiments, device 100 has screens on bothsides, in which the instructions to the user are provided as appropriateon the screen facing the user.

Distance Change

In other embodiments of the invention, the instruction (212) to movemobile device 100 to the new position, includes an instruction to changethe distance between the mobile device and the user (e.g., the user'sface), by moving the device closer to the user or farther from the user.

FIG. 4 is a schematic illustration of distance measurement to an object402, in accordance with an embodiment of the invention.

Optionally, in both the original position (e.g., 404) and the positionafter the change (e.g., 406), front camera 104 acquires an imageincluding a specific body portion (e.g., 410). The instruction (212) tothe user optionally states that the image acquired by front camera 104should include the body portion 410, and possibly includes aninstruction to keep the body portion 410 in a central portion of theimage. Alternatively or additionally, the user is instructed to aim thecamera such that the body portion is located in the same relativelocation within the image acquired after the position change, as beforethe position change. The body portion optionally includes the humancornea, although other body organs having a nearly fixed size fordifferent people may be used. In some embodiments, a plurality ofdifferent body portions may be considered and the distance is determinedbased on a weighted sum of the distance calculated for each bodyportion.

Alternatively or additionally, processor 110 is configured with the sizeof the body portion in an initialization stage and thereafter theconfigured size is used in a plurality of subsequent distancemeasurements performed by the user. The configured size may be enteredmanually by the user, based on a measurement performed without use ofdevice 100, or may be performed using device 100, for example in aconfiguration procedure including acquiring images of the body portionfrom one or more predetermined distances and/or acquiring a plurality ofimages from different distances with known relative locations.

Alternatively or additionally to using a body portion, front camera 104is aimed at an object having a known size, such as a ruler, coin ormoney note. In some embodiments of the invention, the known-size objectis included in the images acquired before and after the position change.In other embodiments, the known-size object is placed by the user nextto the body portion, so that the size of the body portion can bedetermined from the known-size object. In these embodiments, theknown-size object may appear in only some of the images acquired.

The change in position is optionally determined (222) in theseembodiments, based on a comparison of the size of the imaged bodyportion 410 in the front images before and after the position change(Front1, Front2). Optionally, processor 110 counts the number of pixelsover which the cornea (or other specific body organ) spans in imageFront1 and according to the known width of the cornea and the field ofview (iFov) of front camera 104, calculates the distance (F1) betweendevice 100 and the user at the time the image Front1 was acquired.Optionally, the width of the cornea is evaluated as the white to whitecorneal diameter.

Optionally, the distance between front camera 104 and the user (F1) iscalculated as:F1=SizeOfCorneaInMeters/(iFovInRadiansToPixel*SizeOfCorneaInPixels)

Processor 110 optionally additionally determines the size ratio of thecornea and/or other body portions between Front1 and Front2, referred toherein as FrontZoom and the size ratio of one or more features betweenRear1 and Rear2, referred to herein as RearZoom. The one or morefeatures used in calculating RearZoom are optionally selected usingknown methods of feature detection, matching and/or tracking, such asoptical flow, speeded up robust features (SURF) and/or scale invariantfeature transform (SIFT).

The distance to the object from device 100 before the position change isoptionally estimated (224) as:R1=F1*(FrontZoom−1)/(1−RearZoom)

Alternatively or additionally, the calculation includes determining thenew distance (F2) between user 408 and device 100, after the positionchange, in a manner similar to the calculation of F1. F2 may then beused to calculate FrontZoom and/or R2.

Alternatives

Device 100 in some of the above embodiments is assumed to be rigid, atleast in portions on which cameras 104 and 106 are mounted, such thatthe relative positions of the cameras does not change. In otherembodiments, device 100 may be flexible and/or cameras 104 and/or 106are movably mounted on device 100, such that the relative positions ofthe cameras on device 100 may change. In such embodiments, processor 110is optionally configured to receive information on the pose of device100, which pose information includes not only the location andorientation of device 100, but also the relative locations of cameras104 and 106 on device 100.

In the above description, device 100 is assumed to be held by a user andthe instructions as to changing the position of the device are providedto the user. In other embodiments, device 100 is mounted on a tripod,arm or other mount which includes a motor or other device which controlsits movement. The mount is configured to receive movement instructionsdirectly from device 100, such that the entire method of FIG. 2 iscarried out automatically without human aid. Such a setup may be used,for example, for surveillance purposes.

Conclusion

It will be appreciated that the above described methods may be varied inmany ways, including, changing the specific elements used and theirlayout and changing the order of acts in the methods. It should also beappreciated that the above described description of methods andapparatus are to be interpreted as including apparatus for carrying outthe methods and methods of using the apparatus. The present inventionhas been described using non-limiting detailed descriptions ofembodiments thereof that are provided by way of example and are notintended to limit the scope of the invention. Many specificimplementation details may be used.

It should be understood that features and/or steps described withrespect to one embodiment may be used with other embodiments and thatnot all embodiments of the invention have all of the features and/orsteps shown in a particular figure or described with respect to one ofthe embodiments. Variations of embodiments described will occur topersons of the art. Furthermore, the terms “comprise,” “include,” “have”and their conjugates, shall mean, when used in the claims, “includingbut not necessarily limited to.”

It is noted that some of the above described embodiments may describethe best mode contemplated by the inventors and therefore may includestructure, acts or details of structures and acts that may not beessential to the invention and which are described as examples.Structure and acts described herein are replaceable by equivalents whichperform the same function, even if the structure or acts are different,as known in the art. Therefore, the scope of the invention is limitedonly by the elements and limitations as used in the claims.

The invention claimed is:
 1. A method of measuring a distance to anobject, by a device having at least two cameras, comprising: acquiring,by a first camera of the device while the device is in a first position,one or more first images that include the object; acquiring, by a secondcamera of the device while the device is in the first position, one ormore first reference images; after acquiring the one or more firstimages, displaying, to a user: at least one of the one or more firstreference images; a video stream acquired by the second camera, thevideo stream being overlaid on the at least one of the one or more firstreference images, and the video stream directing the user to move thedevice to a second position, the second position being different fromthe first position; an indication of an extent of matching between theone or more first reference images and a current image acquired by thesecond camera, the indication comprising a numerical score thatindicates the extent of matching; and a stop instruction in response tothe numerical score exceeding a predetermined threshold; acquiring, bythe first camera of the device while the device is in the secondposition, one or more second images including the object; acquiring, bythe second camera of the device while the device is in the secondposition, one or more second reference images; determining, based on theone or more first and second reference images, information on adisplacement of at least one camera of the device between the first andsecond positions; and calculating the distance from the device to theobject, based on the one or more first and second images including theobject and the determined information on the displacement of the atleast one camera, the calculation based on a fixed measurement.
 2. Themethod of claim 1, wherein acquiring the one or more first imagesincluding the object comprises acquiring a plurality of images while thedevice is in the first position and generating a single image includingthe object from the plurality of acquired images.
 3. The method of claim2, wherein generating the single image including the object from theplurality of acquired images comprises selecting one of the plurality ofacquired images.
 4. The method of claim 2, wherein generating the singleimage including the object from the plurality of acquired imagescomprises generating a combination of some or all of the plurality ofacquired images.
 5. The method of claim 2, wherein generating the singleimage including the object from the plurality of acquired imagescomprises analyzing the quality of the plurality of acquired images anddiscarding images of a quality below a given threshold.
 6. The method ofclaim 1, comprising displaying to the user instructions directing tochange the position of the device to the second position, afteracquiring the one or more first images.
 7. The method of claim 1,wherein the fixed measurement comprises the size of a body organ andwherein the body organ is included in the one or more first and secondreference images.
 8. The method of claim 7, wherein the body organcomprises a cornea.
 9. The method of claim 1, wherein the fixedmeasurement comprises the distance between the first and second cameras.10. The method of claim 1, wherein the first and second cameras are onopposite sides of the device.
 11. The method of claim 1, wherein the oneor more first images including the object and the one or more firstreference images are acquired in pairs of reference and object images,the reference and object images of each pair being acquiredsubstantially concurrently within less than 1 second.
 12. A device,comprising: a housing; a display; a first camera included in thehousing; a second camera included in the housing; and a processor,included in the housing, configured to: receive one or more first objectimages including an object, acquired by the first camera while thedevice is in a first position; receive one or more first referenceimages acquired by the second camera while the device is in the firstposition; after acquiring the one or more first object images, display,by the display of the device: at least one of the one or more firstreference images: a video stream acquired by the second camera, thevideo stream being overlaid on the at least one of the one or more firstreference images, and the video stream directing a user to move thedevice to a second position, the second position being different fromthe first position; an indication of an extent of matching between theone or more first reference images and a current image acquired by thesecond camera, the indication comprising a numerical score thatindicates the extent of matching; and a stop instruction in response tothe numerical score exceeding a predetermined threshold; receive one ormore second object images including the object acquired by the firstcamera while the device is in the second position; receive one or moresecond reference images acquired by the second camera while the deviceis in the second position; determine a displacement of at least onecamera of the device between acquiring the one or more first objectimages and acquiring the one or more second object images, responsivelyto the one or more first and second reference images; and calculate adistance from the device to the object based on the one or more firstand second object images and the determined displacement, thecalculation based on a fixed measurement.
 13. The device according toclaim 12, wherein the first and second cameras are on opposite sides ofthe housing.
 14. The device according to claim 12, wherein the processoris configured to control the first and second cameras to acquire the oneor more first object and first reference images in pairs of concurrentlyacquired object images and respective reference images.
 15. The deviceaccording to claim 12, comprising a screen located on a same side of thehousing as the second camera.
 16. The device according to claim 12,wherein the processor is configured to provide, after receiving the oneor more first object images and first reference images, instructions onchanging the position of the housing, for acquiring the one or moresecond object and second reference images.
 17. The device according toclaim 12, wherein the processor is configured to analyze an input streamprovided by the second camera relative to the one or more firstreference images, after receiving the one or more first object imagesand first reference images, in order to determine when the housing is ina position suitable for acquiring the one or more second referenceimages and the one or more second object images.
 18. The deviceaccording to claim 12, comprising a screen and wherein the processor isconfigured to display on the screen, after receiving the one or morefirst object images and first reference images, a stream of imagesacquired by the second camera, in order to allow the user to positionthe housing for acquiring the one or more second object and secondreference images.
 19. The device according to claim 12, wherein theprocessor is configured to determine an extent of relative rotationbetween the one or more first and second reference images and use thedetermined extent of relative rotation in determining the displacementof the at least one camera of the device between acquiring the one ormore first object images and acquiring the one or more second objectimages.
 20. The device according to claim 12, wherein the processor isconfigured to determine the displacement by comparing the size of a bodyorgan shown in both the one or more first and second reference images.21. A computer program product for distance estimation, comprising: acomputer readable non-transitory storage medium having computer readableprogram code embodied therewith, which when executed by a computerperforms: receiving one or more first images including an objectacquired by a first camera of a device while the device is in a firstposition ; receiving one or more first reference images acquired by asecond camera while the device is in the first position; after acquiringthe one or more first images, displaying, to a user: at least one of theone or more first reference image; a video stream acquired by the secondcamera, the video stream being overlaid on the at least one of the oneor more first reference images, and the video stream directing the userto move the device to a second position, the second position beingdifferent from the first position; an indication of an extent ofmatching between the one or more first reference images and a currentimage acquired by the second camera, the indication comprising anumerical score that indicates the extent of matching; and a stopinstruction in response to the numerical score exceeding a predeterminedthreshold; receiving one or more second object images including theobject acquired by the first camera while the device is in the secondposition; receiving one or more second reference images acquired by thesecond camera while the device is in the second position; determining,based on the one or more first and second reference images, informationon a displacement of at least one camera of the device between acquiringthe one or more first and one or more second reference images; andcalculating a distance from the device to the object, based on the oneor more first and second images including the object and the determinedinformation on the displacement of the at least one camera, thecalculation based on a fixed measurement.